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Abstract 

In supersymmetry (SUSY) the flavor mixing between top-squark (stop) and 
charm-squark (scharm) induces the flavor-changing neutral-current (FCNC) 
stop decay t\ — ► c\\- Searching for this decay serves as a probe of soft SUSY 
breaking parameters. Focusing on the stop pair production followed by the 
FCNC decay of one stop and the charge-current decay of the other stop, we 
investigate the potential of detecting this FCNC stop decay at the Fermilab 
Tevatron, the CERN Large Hadron Collider (LHC) and the next-generation 
e + e~ linear collider (LC). We find that this decay may not be accessible at 
the Tevatron, but could be observable at the LHC and the LC with high 
sensitivity for some part of parameter space. 
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I. INTRODUCTION 



Flavor-changing neutral-current (FCNC) interactions are strongly suppressed in the 
Standard Model (SM) by the GIM mechanism, which is consistent with the current ex- 
perimental observation. In theories beyond the SM the FCNC interactions are not generally 
suppressed, and thus are subject to stringent constraints from experiments [1]. On the other 
hand, the study of FCNC interactions, especially related to the top quark [2], will play an 
important role in testing the SM and probing new physics. 

Weak scale supersymmetry (SUSY) as a leading candidate for new physics beyond the SM 
provides no further understanding about the origin of flavor. In fact, it extends the mystery 
of flavor by necessarily adding three families of squarks and sleptons. Without additional 
assumptions for flavor structure of the soft SUSY breaking, supersymmetric theories often 
encounter phenomenological difficulties, known as the SUSY flavor problem [3]. This in 
turn implies that there may exist rich FCNC phenomenology. Some highly suppressed 
FCNC processes in the SM may be enhanced in supersymmetric models to a level accessible 
in the future experiments, such as t — > cV (V = ^y,Z,g) and t — > ch [4-6]. On the other 
hand, sfermions may have large flavor mixings via the soft SUSY breaking terms. Even 
if the flavor-diagonality is assumed for sfermions at the grand unification scale, the flavor 
mixings at weak scale are naturally generated through renormalization group equations [7] . 
Therefore, hunting for the exotic FCNC processes predicted by SUSY would be one of the 
important aspects in SUSY searches at the upcoming colliders. 

There have been intensive studies for the FCNC phenomenology in the slepton sector 
[8]. In the squark sector, some interesting FCNC phenomena may arise from the mixing 
between the stop and scharm. On the experimental side, we note that despite of the strong 
constraints on the mixing between first and second generation squarks from K°-K° mixing, 
the mixing between stop and scharm is subject to less low-energy constraints and could 
be maximal [9] although a recent analysis [10] of electric dipole moment of mercury atom 
indicated a nontrivial constraint on the £l — cl mixing. Such a large mixing would reveal 
itself in some processes or subject to some constraints in future collider experiments. On 
the theoretical side, in the Minimal Supersymmetric Standard Model (MSSM) the stop- 
scharm mixing is likely to be large even if there is no mixing at tree level, as first realized 
in [11]. The stop-scharm mixing induces the FCNC stop decay t\ — > ex®, where t\ is the 
lighter one of the stop mass eigenstates and Xi is the lightest neutralino assumed to be 
the lightest supersymmetric particle (LSP). Early searches for this channel at the Tevatron 
experiments have set the bounds <; 120 GeV for m^o ~ 40 GeV [12]. However, if 
kinematically accessible, the tree- level charged-current (CC) decay mode t\ — ► bxt, where 
xt is the lighter chargino, will be most likely dominant although it may be via a three-body 
decay with xl* ~ * [13]. Experimental searches for this mode have also been performed 
at the Tevatron [14], and the current bounds are ^ 135 GeV for mc ~ 80 GeV. In 
their analyses, however, the decay branching fraction of t\ has been simply assumed to be 
100% for each channel under their consideration. There have also been studies [15] on the 
possibility of finding the FCNC stop decay from top quark pair production followed by the 
decay t — > tiXi ~^ c XiXi °f one top and the SM decay of the other top. It is shown that 
such a decay mode, if realized in the tt pair events with a substantial branching fraction, 
could be observable in some part of the SUSY parameter space at Run 2 of the Tevatron 
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collider [16]. 

In this article, we focus on the direct stop pair production followed by the FCNC decay 
of one stop (ti — > cx°) and the charge-current decay of the other stop (ii — > bxt)- We allow 
arbitrary branching fractions for these two channels. By simulating both the signal and the 
SM backgrounds, we examine to what levels the branching ratio and the stop-scharm mixing 
parameter can be probed at the Tevatron collider, the CERN Large Hadron Collider (LHC) 
and the 500 GeV next-generation e + e~ linear collider (LC). 



II. STOP-SCHARM MIXING AND FCNC STOP DECAY 



Following Ref. [11], we start in the framework of the MSSM and assume that the tree 
level interactions are flavor diagonal for stops and scharms. The flavor mixing between 
stops and scharms is then induced via loops. The dominant effects are from the logarithmic 
divergences caused by soft breaking terms. Such divergences must be subtracted using a 
soft counter-term at the SUSY breaking scale, such as the Plank scale M p in supergravity 
(SUGRA) models. Thus a large logarithmic factor (1/16-7T 2 ) ln(Mp/m^) 0.5 remains after 
renormalization 1 . In the approximation of neglecting the charm quark mass, Cr does not mix 
with stops. The mixing of cl with stops result in the physical states given approximately 
by 
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9 t is the mixing angle 2 between left- and right-handed stops, defined by 
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For some mechanisms of SUSY breaking other than gravity mediation, such as gauge mediation, 
the SUSY breaking scale can be much lower than the Plank scale and thus this factor may be 
smaller. 

2 Note that our definition of 0t differs from that in Ref. [11] by a minus sign. 



3 



In the above, we have adopted the notation in Ref. [17], with < nip 



and Mjj are soft-breaking mass terms for left-handed squark doublet Q, right-handed down 
squark D and Higgs doublet Hi, respectively. is the coefficient of the trilinear term 
HiQD in soft-breaking terms and tan/5 = v-ijv\ is ratio of the vacuum expectation values 
of the two Higgs doublets. Note that cl is a mass eigenstate in our analysis since we do not 
consider the mixing between c L and cr, which is proportional to the charm quark mass. 

From the above equations, we note that besides the large logarithmic factor \a.{M'^/m\ v ), 
the mixings are proportional to tan 2 f3 and thus can be further enhanced at large tan (3. If 
we assume that all soft SUSY-breaking parameters are of the same orders in magnitude, we 
then have typically e m 0.01(tan/3/10) 2 and thus e is much smaller than unity. (Note that 
to make the approximate expansion of Eq. (1) valid, e should be much smaller than unity.) 
Without such an assumption, e can be larger because in the sum Mq + M|, + Mj^ + \A d \ 2 

only Mq is related to stop and scharm masses while other parameters are independently free 
in the MSSM. 

The stop mass is particularly important for our study and will be retained as a free 
parameter in our numerical calculations. The lightness of the stop is quite well motivated 
in some SUSY models like SUGRA and is also preferred by electroweak baryogenesis [18]. 
On the other hand, the current lower bound on its mass is about 135 GeV [14], albeit under 
some assumptions. We will thus explore the mass range 



150 GeV < m h < 250 GeV 



(6) 



where the upper end is the kinematic limit for a 500 GeV linear collider. So we assume an 
upper bound of about 250 GeV in our numerical analysis. 

The flavor mixing between stop and scharm will induce the FCNC stop decay t\ — > cXi- 
Since the charge- current decay ti — > bxt can be the other important decay mode, the 
branching ratio of the FCNC decay is obtained by 



BF = 
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Here, JV|- denotes the matrix element projecting the i-th neutralino into photino (j = 1), 
zino (j = 2), and two neutral Higgsinos (j = 3,4). Vij is the matrix element projecting 
the i-th left-handed chargino into wino (j = 1) and the charged Higgsino (j = 2). The 
gaugino masses and mixing are determined by the soft SUSY-breaking parameters Mi, M 2 , 
as well as /i, tan/3. There are strong theoretical motivations to further constrain these 
parameters [19]. First of all, the supergravity models predict the unification relation Mi = 
|M 2 tan 2 6'vi/ ~ 0.5M 2 . Radiative electroweak symmetry breaking generally yields a large \i 
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parameter, although the naturalness arguments prefer a lower value of [i. This scenario leads 
to the LSP Xi bino-like, and xt wino-like, which is also favored for a SUSY dark matter 
interpretation. Regarding the other parameter tan /3, the LEP experiments excluded small 
values tan/3 < 2 [20]. For the sake of illustraton, we thus choose the following representative 
set of parameters 

M 2 = 150 GeV, n = 300 GeV, tan/3 = 10. (10) 
The chargino and neutralino masses in units of GeV are then given by 

= 133, = 328, 

X-l X-2 

m^o = 72, m^o = 134, m^o = 308, m^o = 327. (11) 

In our analysis the chargino xt must be lighter than stop t±. Such a light chargino decays 
into ff'Xi if is a quark or lepton) through exchanging a W-boson, or a charged Higgs 
boson, a slepton, a squark [21]. Since, typically, the charged Higgs, sleptons and squarks are 
much heavier than the IF-boson, such decays occur dominantly through the IF-exchange 
diagram and the branching ratio for the clean channels xt ~^ £ + vx° — e and //) is thus 
approximately 2/9. 

With the parameters in Eq. (10), the branching fraction B{t\ — * CX\) in the no mixing 
limit is approximately given by 

fl.3|e| 2 , for m~ tl =150 GeV, 
at ~ \ 0.16|e| 2 , for m k = 250 GeV. 

For a lighter m^, the decay t\ — > bxt is kinematically suppressed; and for a heavier m^, 
this charged-current channel becomes dominant. 

Note that our choice of parameters in Eq. (10) is rather representative for which the 
decay modes t\ — > cxi and bxt are both kinematically accessible. The exception is in the 
Higgsino-like region (M 2 > In this case, both the LSP and xt are mainly Higgsino-like, 
and are about degenerate in mass close to \i. The lepton produced in the decay xt ~^ x\^ v 
will be too soft to be experimentally identifiable, making the signal difficult to observe. As 
we indicated earlier, this situation is disfavored by the arguments of SUSY-GUT and dark 
matter. We will thus not pursue this special case further. 



III. OBSERVABILITY OF FCNC STOP DECAY AT COLLIDERS 

Since the stop t\ is likely to be significantly lighter than any other squark and thus the 
production rate of is larger than other squark pairs, as well as than t\ti or t^-, we only 
consider the production of t\i\ in our analysis. Inclusion of the channels t\ti and t 2 ^2 would 
enhance the signal observability although the kinematics of the final states would be more 
involved to study. For a light stop with a mass close to the top quark, the QCD corrections 
enhance the total cross section of stop pair by a factor of about 1.2 at the Tevatron energy 
and 1.4 at the LHC energy [22]. This enhancement (the so-called K factor) will be taken 
into account in our calculation. The one-loop corrections to stop pair production in a 500 
GeV e + e~ collider were found to increase the cross section by 10-20% [23] and we assume 
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an enhancement factor K = 1.1 in our analyses. Going beyond the crude assumption 
on the branching fractions of the t\ decay in the Tevatron studies [12,14], we consider 
the FCNC decay of one stop t\ — > cx±, and the charge-current decay of the other one, 
t\ — > bxf — > M + uxi- The signal we are proposing to look for is a t\ii event giving rise 
to an energetic isolated charged lepton (e or //), a 6-quark jet, a (charm) jet and missing 
transverse energy, denoted by jb£ +$ T . 

First, we consider the search at hadron colliders. To simulate the acceptance of the 
detectors, we impose some kinematical cuts on the transverse momentum (pr), the pseudo- 
rapidity (77), and the separation in the azimuthal angle-pseudo rapidity plane (AR = 
yj (A0) 2 + (Ar]) 2 ) between a jet and a lepton or between two jets. We choose the basic 
acceptance cuts for the Tevatron 

p e T , pfi, $ T > 20 GeV, 

Viet, Vi < 2-5, (13) 
ARjj, AR je > 0.5. 

We increase the threshold for the LHC as 

p e T > 20 GeV, jj? > 35 GeV, > 30 GeV, 

Viet, Vt < 3, (14) 
ARjj, AR je > 0.4. 

Furthermore, we assume the tagging of a 6-quark jet with 50% efficiency and the probability 
of 0.4% (15%) for a light quark (c-quark) jet to be mis-identified as a 6-jet. 

To make the analyses more realistic, we simulate the energy resolution of the calorimeters 
by assuming a Gaussian smearing on the energy of the final state particles, given by 

for leptons, (15) 

for hadrons, (16) 

where E is in GeV. 

The potential SM backgrounds at hadron colliders are 

1) bq(q)^tq'(q'); 

2) qq' -> W* -> tb; 

3) Wbb; Wcc; Wcj; Wjj; 

4) ti^W~W + bb; 

5) gb - tW; 

6) qg — > q'tb. 
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Tevatron 2 TeV 


LHC 14 TeV 


basic cuts 


basic + rriT 


basic cuts 


basic + rriT 


signal 


23 


6.6 


720 


220 


qb — > g'i 


120 


5.0 


7400 


400 


— ► £6 


39 


2.3 


280 


17 




130 


2.5 


570 


45 


Wcc 


80 


1.5 


450 


36 


Wcj 


670 


3.8 


7600 


650 


Wjj 


500 


2.9 


1700 


150 


tt 


7.9 


3.8 


600 


300 



TABLE I. Signal t\t\ — > Ibcfij, and background cross sections in units of fb. The signal results 
were calculated by assuming = 150 GeV and other parameters are in Eq. (10). The charge 
conjugate channels have been included. The signal results do not include the branching fraction 
factor 2BF(1 — BF), which should be multiplied to obtain the actual signal rate for a given value 
of BF. 



The backgrounds 5) and 6) are of modest rates and can mimic our signal only if the extra jet 
is missed in detection. After vetoing the extra central jet, these backgrounds are effectively 
suppressed. The tt background 4) is of a large production cross section, especially at the 
LHC. It can mimic our signal if both W's decay leptonically and one of the charged leptons 
is not detected, which is assumed to occur if the lepton pseudo-rapidity and transverse 
momentum are in the range \r)(£)\ > 3 and Pt(£) < 10 GeV. In addition, we also have 
some SUSY backgrounds. In case of i\ being significantly lighter than other squarks, the 
dominant SUSY background is the pure charged-current decay t\ti — > XiXibb. Also, at the 
high end of the top squark mass range considered in our analysis, t\ can decay to t* + Xi or 
even t + Xi- All these processes give a tt-like signature [24] and can mimic our signal just 
like the SM production of tt. However, compared with tt background 4), such backgrounds 
are much smaller since their production rates are much lower than the tt background. 

We notice that for most of the background events the missing energy comes only from 
neutrinos in W decay, while for the signal the missing energy contains the extra contribution 
from the neutralinos. From the transverse momentum of the lepton p\ and the missing 
transverse momentum j$ T , we construct the transverse mass as 

m T = vfel + bM) 2 -G4 + fc) 2 - (17) 

For the background events where the only missing energy comes from a neutrino from W 
decay, rriT is always less than M w (and peaks just below M w ) without energy smearing. 
Smearing pushes some of the events above Mw- For the signal wit is spread out widely above 
and below Mw, due to the extra missing energy of the neutralinos. In order to substantially 
enhance the signal-to-background ratio (S/B), we apply a cut 

m T > 90 GeV. (18) 
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160 180 200 220 240 

Stop Mass (GeV) 

FIG. 1. Discovery limits of the branching ratio of the FCNC stop decay t\ — ► ex? versus the 
stop mass. The region above each curve is the corresponding observable region. 

We first present the signal and background cross sections at the Tevatron (2 TeV) and 
LHC (14 TeV) under various cuts in Table I. One sees that with only the basic acceptance 
cuts, the various SM backgrounds can overwhelm the signal. The implementation of the ttit 
cut reduces the backgrounds Wbb, Wcc, Wjj and Wcj efficiently. The production rate of 
the signal b+}et+£ $ T can be obtained by multiplying the tiii cross section by the branching 
fraction factor (2/9)2BF(l — BF) for a given value of BF, Our results for the 2a sensitivity 
on BF at the Tevatron versus the stop mass are shown in Fig. 1 (the top curve). We find 
that due to limited statistics, Tevatron Run 2 with a luminosity of 2 fb" 1 is not able to 
discover the signal nor even set any significant bounds on the branching fraction BF. A 
bound at 2a level could be reached at the Tevatron energy with a luminosity of 20 fb _1 for 
< 180 GeV, corresponding to BF ~ 20%. At the LHC the 5<r-discovery is accessible, 
reaching the branching fraction below 1% even for a low luminosity 100 fb _1 . From Fig. 1 
one sees that the detection sensitivity for hadron colliders does not monotonously increase as 
the stop mass decreases. Instead, when the stop becomes too light, the detection sensitivity 
decreases. This is the effect of the cuts applied in our simulation and can be understood 
as follows. As the stop mass decreases, the stop pair production rate increases. However, 
when the stop becomes too light, the 6-jet from t\ — > xfb becomes very soft and thus failed 
to pass the selection cuts so that it decreases the detection sensitivity. 

The results at the LHC are obtained by applying the basic and the tut cuts. The signal 
significance is obtained in terms of Gaussian statistics, given by the signal and background 
events S/VB. Although the sensitivity reach at the LHC is impressive as shown in Fig. 1, 
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Stop Mass (GeV) 

FIG. 2. 5cr discovery limits of the stop-scharm mixing parameter e versus the stop mass with 
the stop mixing angle 6% = tt/W. The region above each curve is the corresponding observable 
region. 



the signal-to-background ratio S/ B becomes rather low when reaching the small branching 
fraction. Thus the sensitivity relies on the successful control of the systematics in the 
experiments. 

It is known that the experimental environment is much cleaner at an e + e~ collider. Now 
we recapitulate our analyses for an e + e~ linear collider with C. M. energy of 500 GeV. 
Since the environment of e + e~ colliders is much cleaner, we will evaluate the production 
rate of the signal b+jet+i F, T simply by multiplying the cross section a(e + e~ — > t\i{), the 
branching ratio (4/9)BF(l — BF), the 6-tagging efficiency assumed to be 50%, and the 
detection efficiency of kinematics assumed to be 80%. The possible SM backgrounds are 



e+e" -> W + W~ -> jj'i 
e+e" -> tt -> bW + bW~ 



bbjj'l 



(19) 
(20) 



However, these backgrounds can be effectively separated due to the rather different kine- 
matical features from the signal. If we define the recoil mass as 



obs ) 



(21) 



where the sum is over all momenta of the observed final state particles, then we notice that 
the backgrounds have rather small recoil mass from the single missing neutrino. The recoil 
mass for the signal on the other hand is quite large since the neutralino is very massive. 
Similar to the case of hadron colliders, the dominant SUSY background is t\i\ — ► xfxibb, 
which is small and neglected in our numerical analysis. For a stop mj ^ 230 GeV when 
the threshold is sufficiently open for ^/s = 500 GeV, one can reach a 5<r observation at the 
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Stop Mixing Angle (degree) 

FIG. 3. 5a discovery limits of the stop-scharm mixing parameter e versus the stop mixing angle 
6t for mj = 170 GeV. The region above each curve is the corresponding observable region. 

LC with a branching fraction of 1-5% even for a luminosity of only 100 fb _1 , as shown in 
Fig. 1. 

The exclusion and discovery limits of the branching fraction can be translated into the 
limits on the stop-scharm mixing parameter e, which can be predicted for a specific SUSY 
model. At this stage, the stop mixing angle 9 t needs to be specified. For illustration, we first 
fix 9 t = 7T/T0 and the resulting limits on e are shown in Fig. 2, corresponding to the results 
of Fig. 1. For stop mass of 150 GeV the 5a discovery limit with a luminosity of 100 fb _1 is 
e > 0.09 at the LC, and e ^ 0.20 at the LHC For a heavier stop, the detection sensitivity 
at the LC drops much more rapidly than that at the LHC due to the limited CM. energy 
of the LC. Note that although Run 2 with 20 fb _1 has a 2a sensitivity (as shown in Fig. 1) 
to the decay branching ratio, it has no sensitivity to e < 0.3. When the 2a sensitivity limits 
of Run 2 in Fig. 1 are translated to the mixing parameter e, rather large e values (k, 0.5) 
are obtained. A large e is not theoretically favored, as implied in Eqs. (1 — 4). 

The obtained limits on e are sensitive to the mixing angle 9 t , which controls the partial 
width for the CC decay as seen in Eq. (9). The dependence of e limits on 9 t is shown in Fig. 3 
for a fixed value of stop mass m~ tl = 170 GeV. For the mixing angle to be near a certain 
value, tan# t ~ (\/2m w sin/3/m i )(Vii/Vi 2 ), the CC mode is suppressed and the sensitivity 
to the FCNC mode is greatly enhanced. For our choice of the SUSY parameters, this occurs 
near 9 t ~ 9°. The 5a sensitivity with 500 fb" 1 for the LHC and LC could reach as far as 
e ~ 0.01. For nearly maximal mixing 9 t ~ 45°, on the other hand, the sensitivity could be 
reduced to about e ~ 0.4. 

Furthermore, the limits or observation of the mixing parameter e can be translated 
into some knowledge on certain soft SUSY breaking parameters. From Eqs. (2) and (3), 
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we see that the mixings are proportional to a sum of certain parameters, typically like 
(Mq + M|) + Mft i + \A d \ 2 ) / (m 2 — rrv~ L ). This can vary independently of mi and since 
only Mq in this sum is related to and rric L - For the purpose of illustration, taking 
mi x = 150 GeV, 9 t = n/10, rric L = 200 GeV and other SUSY parameters given in Eq. (10), 
we obtain the LHC discovery (5<r) limit with a luminosity of 100 ftr 1 

y/M^ + M 2 D + M 2 Hl + \A d \ 2 + 0.3m t A* d > 1.4 TeV, (22) 
or, in case of non-observation, the 2a bound given by 

^M 2 + M 2 d + M 2 Hi + \A d \ 2 + 0.3m t A* d & 1.0 TeV. (23) 

Due to the nature of the multiple parameters as a combination involved in the expression, 
more comprehensive analyses would be needed, possibly to combine with other experimental 
knowledge on the SUSY parameters, in order to extract the information for the theory 
parameter space. 

IV. CONCLUSIONS 

In summary, we studied the potential of detecting the FCNC stop decay t± — > cxi, as a 
probe of stop-scharm mixing, at the upgraded Tevatron, the LHC and the LC. Rather than 
performing an exhaustive scan of the SUSY parameters, we chose a representative set of the 
relevant parameters to demonstrate the possibility of observation. Through Monte Carlo 
simulation, we found that the signal at the Tevatron is too weak to be observable for the 
choice of well- motivated SUSY parameters. At the LHC on the other hand, with judicial 
kinematical cuts, it is quite possible to observe a 5a signal with a branching fraction as 
low as 1% even for a luminosity of 100 fb -1 . However, it should be noted that systematic 
effects in the experiments must be under control. At an LC of ^Js = 500 GeV, one can 
reach a 5<r observation with a branching fraction of 1 — 5% for a luminosity of 100 fb -1 . The 
limits or observation of this important decay mode can be translated into some knowledge 
on certain soft SUSY breaking parameters. We finally note that in our study we have chosen 
a representative scenario for relevant SUSY parameters in which the lightest neutralino and 
chargino are gaugino-like. In the region of SUSY parameters where the lightest neutralino 
and chargino are Higgsino-like, the signal would be more difficult to observe. 
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